(12) United States Patent 

Eintracht et al. 



IllliUlilll^^ 

US006687878B1 

(10) Patent No,: US 6,687,878 Bl 
(45) Date of Patent: Feb. 3, 2004 



(54) SYNCHRONIZING/UPDATING LOCAL 
CLIENT NOTES WITH ANNOTATIONS 
PREVIOUSLY MADE BY OTHER CLIENTS 
IN A NOTES DATABASE 

(75) Inventors: Zvika Eintracht, Ramat Gan (IL); 

Alexander Ovsiankin, Lod (IL); 
Roman Dementiev, Givatayim (IL); 
Gil Sideman, Raanana (IL) 

(73) Assignee: Real Time Image Ltd., Or Yehuda (IL) 

( * ) Notice: Subject to any disclaimer, the term of this 
patent is extended or adjusted under 35 
U.S.C. 154(b) by 0 days. 

(21) Appl. No.: 09/270,255 

(22) Filed: Mar.J5, 1 999 

(51) Int. a: G06F 7/00 

(52) U.S. CI 715/512; 513/515 

(58) Field of Search 707/501, 512, 

707/513, 515; 715/501.1, 512, 513, 515 

(56) References Cited 

U.S. PATENT DOCUMENTS 

5,165,012 A 11/1992 Crandall el al 345/347 

5,220,657 A ♦ 6/1993 Bly et al 711/152 

5,231,578 A 7/1993 Levin et al 707/512 

5,392,400 A 2/1995 Berkowitz et al 709/203 

5,671,428 A 9/1997 Muraaaga et al 345/329 

5,806,079 A 9/1998 Rivette et al 707/512 

5,821,931 A 10/1998 Berquist et al 345/346 

5,831,615 A * 11/1998 Drews et al 345/344 

5,832,474 A * 11/1998 Lopresti et al 707/2 

5,838,914 A 11/1998 Carleton et al 709/204 

5,845,301 A * 12/1998 Rivette et al 707/512 

5,860,074 A 1/1999 Rowe et al 707/526 

5,870,547 A 2/1999 Pommier et al 709/204 

5,870,759 A * 2/1999 Bauer et al 707/201 

5,890,177 A ♦ 3/1999- Moody et al 715/511 

5,966,512 A ♦ 10/1999 Bates et al 709/205 

6,052,514 A ♦ 4/2000 Gill et al 345^33 



6,052,695 A ♦ 4/2000 Abe et al 707/202 

6,081,291 A * 6/2000 Ludwig, Jr 348/16 

6,081,829 A ♦ 6/2000 Sidana 709/203 

(List continued on next page.) 

OTHER PUBLICAnONS 

Trigg et al, "Supporting Collaboration in Notecards", Pro- 
ceedings of the 1986 ACM conference on Computer-sup- 
ported cooperative work, Dec. 1986, pp. 153-162.* 

Primary Examiner — Stephen S. Hong 

Assistant Examiner — Cesar B. Paula 

(74) Attorney, Agent, or Firm — Howard Zaretsky 

(57) ABSTRACT 

A system for collaborative document annotation whereby 
notes (i.e. annotations) associated with a document, such as 
an image or text document, are s tored in a notes database_o n 
a c entral notes serv er. The documents and associated anno- 
tations are treated independently from each other whereby 
separate, data structures are created for the documents and 
for the associated annotations. A web server application on 
the server side functions to capture requests from one or 
more note client applications for creating, storing, editing 
an d_jetrieving an no tations related to specific documents 
stored on the notes server. 6n the client side, the notes client 
funciions to ciisplay'fhe document that the user wishes to 
annotate and provides the tools necessary to permit the user 
to create, edit, delete, retrieve and store notes. A synchro- 
nization process transmits the annotations generated by the 
user from the notes client to the notes server. In response, the 
notes server transmits back an acknowledgement along with 
any new notes that other notes clients may have posted since 
the last synchronization was performed thus enabling mul- 
tiple notes clients to annotate a document asynchronously 
with respect to each other. When annotations are posted to 
the notes server by a notes clie nt. Ihe state of tb^ gnnr||^||Yip 
d aiaPase is synch ronized such that all other notes clients can 
retrie ve the curr ent, up to date anaalaLifiiis.associatedj^Kith a 
document. 
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SYNCHRONIZING/UPDATING LOCAL 
CLIENT NOTES WITH ANNOTATIONS 
PREVIOUSLY MADE BY OTHER CLIENTS 
IN A NOTES DATABASE 

FIELD OF THE INVENTION 
The present invention relates generally to collaborative 
work systems and more particularly relates to a system for 
collaborative document annotation based on the exchange of 
notes attached to web based documents, the system being 
suitable for use over a network such as the Internet based 
World Wide Web (WWW). 

BACKGROUND OF THE INVENTION 

A basic function of World Wide Web browsers in common 
use today, such as Netscape Navigator and Microsoft Inter- 
net Explorer, is to allow web pages stored in remote Web 
server to be retrieved and viewed. The user initiates a 
viewing session on a web browser by specifying a particular 
URL corresponding to a page to be viewed. The web server 
transmit the various elements of the requested page to the 
browsing application. Any special data formats, e.g., video, 
certain images, documents not written in HTML language, 
etc., are treated by one or more web browser software 
plug-ins on the client computer. 

A common need is to be able to view the document, i.e., 
images, etc., on the client computer and be able to add one 
or more annotations to the document. It is becoming more 
and more common for groups of two or more people to work 
collaboratively on a project. The members of the group may 
or may not be located in the same geographic area. The 
collaboration is often via a network such as the global 
Internet. The members of the group may be located any- 
where in the world as long they have a connection to the 
Internet. 

The ability to annotate documents displayed on screen is 
very useful when information needs to be exchanged among 
more than one person. It would be beneficial if one persons's 
comments could be shared among many others by attaching 
notes that travel with the imderlying document being worked 
on. 

Currently, however, it is not possible to annotate docu- 
ments on the client side such that other clients can see the 
notes. The entire document with the attached notes must be 
transmitted to other clients for them to see the notes. 
Transmitting the entire document from client to client 
through the server in order to exchange annotation is 
burdensome, inefl5cient and costly in terms of time and 
bandwidth. 

The majority of prior art systems that with image and page 
annotation are adapted to handle annotation only on the 
client side. In these systems, a user launches an editing tool 
to open a document locally on her/his computer. The docu- 
ment is edited locally with the other chents not being able to 
easily view the changes or notes. Some of these editing tools 
permit attaching data information such as annotations to the 
document. These systems operate by saving the annotations 
within the local file system that is also used to store the 
document. 

Also known in the prior art are systems that permit 
collaboration that is based on the replication of the display 
at one chent system to the display on another client system. 
These systems, however, require that all parties to the 
collaboration session be logged on to the server at the same 
time in order to view the annotations made by one or more 
of the parties. 
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Thus, there is strong felt need for a system that would 
allow multipartly collaboration based on the asynchronous 
exchange of annotations over a network such as the Internet 
without the requirement that all parties wishing to collabo- 
5 rate be simultaneously logged on to a server. 

SUMMARY OF THE INVENTION 

The present invention is a system for collaborative docu- 
ment annotation whereby notes or annotations associated 

10 with a document, such as an image or text document, are 
stored on a web server. The documents and associated 
annotations are treated independently from each other. Sepa- 
rate data structures are created for the documents and for the 
associated annotations thus permitting their independent 

15 management. 

The invention can be implemented as software, a portion 
of which executes on the server side and a portion that 
executes on the client side. The server side may comprise a 
plurality of software applications running in parallel that in 
combination provide the server functionality of the inven- 
tion. Aweb server apphcation on the server side functions to 
capture special requests from one or more client applications 
for creating, storing, editing and retrieving, annotations 
related to specific documents located in the server. A notes 
server functions to log all annotation activities along with 
information about the corresponding clients that create, edit 
and retrieve them. 

The notes information are transmitted between client and 
server applications via TCP/IP protocols over communica- 
tions means such as a LAN, WAN or the Internet. The 
invention may be adapted to operate over an Intranet, e.g., 
LAN, or Extranet, e.g., WAN or the Internet. 

On the client side a client software application functions 

2^ to display the document that the user wishes to annotate and 
provides the tools necessary to permit the user to create, edit, 
retrieve and store notes. Note that the client software apph- 
cation can be implemented as a web browser plug-in mod- 
ule. The plug-in contains the user interface for navigating 
within the document and for handling the notes. 

A synchronization button is provided which, when 
pressed by the user, transmits the annotations generated by 
the user from the client to the server using a particular 
protocol. In response, the server transmits back an acknowl- 

45 edgement along with any new notes that other clients may 
have posted since the last synchronization was performed. 

A user may annotate many documenLs at the same time by 
opening several web browser windows. In addition, other 
clients can annotate either the same document or other 

50 documents at the same or a later time. In accordance with the 
invention, the annotations are transmitted from the server 
independent of the data transmitted that is related to the 
viewed document. At the client side, the client application 
layers the annotations over the image (or document) in 

55 accordance with the coordinates of each. 

As a result, the present invention has the advantage of 
permitting multiple clients to annotate a document that 
resides on a central web server in an asynchronous fashion. 
When annotations are posted to the server by a client, the 

60 state of the annotation database is synchronized such that all 
other clients can retrieve the current, up to date annotations 
associated with a document. As an option, the server can be 
adapted to trigger one or more alarms in response to a 
change being made to the database maintained by the server. 

65 lliose clients that wish to be notified in the event of a 
database change at the server, can be notified by means of 
visual indications, audio and/or e-mail. In addition, the 
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alarms can filtered in numerous ways. Examples of filtering rection services so as to provide user level security for 

include, but are not limited to, specific documents, docu- documents and notes stored on the server. The security 

ment types, document folders, annotation contents and spe- means comprises a Document Location Table and a Client 

cific clients. Access Key Table. 

The system of the present invention further allows anno- 5 The document comprises an image or a document con- 

tations to be kept private. Annotations can be kept secure sisting of a page description language. The notes client is 

from changes made by non authorized clients. The time an adapted to permit the user to annotate the document in 

annotation was created is also tracked and stored in terms of parallel with the display of the document. The document 

the clock in the server. When the time stamp of the anno- comprises a plurality of pages whereby each page can be 
lation is shown on the client, it is shown in terms of the clock lO annotated independently of all the other pages. The synchro- 

on the client. nization means is user initialed or is triggered by the notes 

There is provided in accordance with the present inven- via the signaling of selected notes clients, 

tion a system for annotating documents comprising a docu- There is also provided in accordance with the present 

ment file located on a server, the document file for storing invention method of annotating documents, the method 

one or more documents, a notes database located on the .comprising the steps of getting a first request comprising 

server, the notes database for storing one or more notes, each Universal Resource Lx)cator (URL) associated with a docu- j 

note associated with a particular document, one or more ment to be viewed and annotated and transmitting the first/ 

notes clients coupled to a network, each notes client opera- request from a client to a server, sending a first response 

live to locally display a representation of a document from the server in response to the first request, the first 

remotely stored on the server in the document file, the notes response comprising the document type of and a represen- 

client adapted to permit a user to annotate the document with tative of the requested document, getting a second request 

one or more notes, the notes client operative to simidta- comprising a document note URL corresponding to the notes 

neously display the one or more notes associated with the associated with the document and sending the second 

document over the displayed document such that the docu- request from the client to the server, sending a second 

ment is viewable along with the one or more notes, a notes response from the server in response to the second request, 

server coupled to the network, the notes server operative to the second response comprising one or more notes associ- 

slore the documents in the document file separately from ated with the document, displaying the representation of the 

notes stored in the notes database, the notes server receiving document and the associated one or more notes locally on 

one or more notes associated with a particular document the client such that both are viewable simultaneously, per- 

from the one or more notes clients and synchronization mitting the user to perform annotation functions on the client 

means within the notes clients and the notes server, the so as to generate a third request comprising one or more note 

synchronization means for updating the notes server with events related to the document that are transmitted from the 

any notes events processed by the notes clients and for client to the server and sending a third response from the 

updating the notes client with the results of synchronization server to the client in response to the third request, the third 

updates, previously performed by other notes clients since response comprising an updated list of notes associated with 

the last synchronization event. the document, the list of notes incorporating any note events 

The notes database comprises a Notes Table for storing previously received from other clients, 

the current content of a note and a Notes Log Table for The step of getting a first request comprises the step of 

storing a transaction history of all not events for notes querying the user for a user name and password. The step of 

associated with a particular document. The notes client sending a first response comprises the step of authenticating 

comprises a display, network communication means, a pro- the user on the server before sending the first response. The 

cessor suitably programmed to run a web browser and a method further comprises the step of getting a fourth request 

client note plug-in adapted to run within the web browser. comprising a folder URL associated with the folder com- 

he network comprises the Internet, a TCP/IP network or prising one or more documents to be viewed and annotated 

/n Intranet. The notes server comprises network communi- and transmitting the fourth request from the client to the 

□ation means, a processor suitably programmed to provide a server. 

veb server services, provide notes agent services, provide The method further comprises the step of sending a fourth 

document and notes browsing and synchronization services response from the server in response to receiving a request 

Ao the one or more notes clients. The notes agent services comprising a folder URL, the fourth response comprising a 

(comprises an Internet Server Application Programming directory listing of the folder. The document comprises an 

Interface (IS API). image or a document represented using a page description 

The system further comprises a notes administrator for language, 

providing one or more Universal Resource Locations (URL) BRIEF DESCRIPTION OF THE DRAWINGS 
aliases in managing and adminstrating the notes server and 55 

a note viewer for generating reports on the history of The invention is herein described, by way of example 

annotation on a per user or a per document basis that have only» with reference to the accompanying drawings, 

occurred over a period of time. wherein: 

llie system permits two or more user to collaboratively FIGS. lA, IB, IC are illustrations of an example image 
work on the same document whereby each user is capable of 60 displayed with and without multiple annotation displayed 

viewing annotations made by the other users upon synchro- thereon; 

nization with the notes server. A single page of a multipage FIG. 2 is an illustration of an example annotation; 

document can be annotated on a notes client without requir- FIG. 3 is a block diagram illustrating the collaborative 

ing the entire multipage document to be transmitted to the document annotation system of the present invention; 
notes client. 65 FIG. 4 is a flow diagram illustrating the initiahzation 

The system further comprises security means for provid- portion of the collaborative document annotation system of 

ing Universal Resource Locator (URL) translation and redi- the present invention; 
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DETAILED DESCRIPTION OF TIIE 
INVENTION 

Notation Used ITiroughout 
The following notation is used throughout this document. 



Term 


Definition 


GMT 


Greenwich Mean Time 


HTML 


Hypertext Text Markup Language 


HTTP 


Hypertext Transport Protocol 


lis 


Internet Information Server 


IP 


Internet Protocol 


ISAPI 


Internet Server Application Programming Interface 


LAN 


Local Area Network 


MIME 


Multipurpose Internet Mail Extension 


NFS 


Network File System 


PDF 


Portable Document Format 


RAM 


Random Access Memory 


RGB 


Red Green Blue 


SMTP 


Simple Mail Transport Protocol 


SQL 


Structured Query Language 


TCP 


Transport Control Protocol 


TIFF 


Tag Image File Format 


URL 


Universal Resource locator 


WAN 


Wide Area Network 


WWW 


World Wide Web 



The following terminology and definitions apply through- 
out this document. 



•Term 



Definition 



Annotation A portion of text or a graphical drawing that is associated 
(Note) with a specific location in a document 

Annotation The time period during which a user creates, edits, 
Session retrieves and stores one or more notes associated with 

a document. 



10 



FIG. 5 is a message flow diagram illustrating the trans- 
action sequence between client and server during initializa- 
tion; 

FIG. 6 is a flow diagram illustrating the document brows- 
ing portion of the collaborative document annotation system 
of the present invention; 

FIG. 7 is a message flow diagram illustrating the trans- 
action sequence between client and server during document 
browsing; 

FIG. 8 is a flow diagram illustrating the note synchroni- 
zation portion of the collaborative document annotation 
system of the present invention; 

FIG. 9 is a message flow diagram illustrating the trans- 
action sequence between client and server during note 15 
synchronization; 

FIG. 10 is a diagram illustrating the Client Annotation 
Event Data Structure in more detail; 

FIG. 11 is a diagram illustrating the note data structure of 20 
the Qient Annotation Event Data Structure of FIG. 10 in 
more detail; 

FIG. 12 is a diagram illustrating the Server Annotation 
Response Data Structure in more detail; and 

FIG. 13 is a diagram iUustrating the note data structure of 
the Server Annotation Response Data Structure of FIG. 12 
in more detail. 



-continued 
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Term 



Definition 



Note Alarms 

Note Anchor 

Note Client 

Note Contents 
Note 

Document ID 
Note Event 



Note 

Owner ID 
Note 

Persistence 

Note Serial 
Number 
Note Server 

Note 

Synchroniza- 
tion 

Note Time 
Stamp 
Notes 
Database 
Notes Log 



Notifications that the Notes Server received a Note 
Event. 

The specific location in a document associated with a 
note. 

Hie application that displays the annotated document 
and allows for annotation sessions. 
The text or graphical information contained in the note. 
Identifier for documents and sets of aggregated 
documents organized under the same logical structure. 
A message related to a note that did not exist at the time 
of the last synchronization with the Notes Server, or a 
note that was modified or deleted since that time. 
Identifier used to differentiate between notes generated 
by various users. 

Mechanism whereby the Notes Clients saves the Note 
Events to the local file system, allowing for 
synchronization at a later time. 
A number uniquely identifying a note. 

The application that provides the centralized 

role of document and note management. 

The update activity whereby one or more Note Events 

are exchanged between the Notes Server and 

the Notes Client. 

The time stamp saved on the Notes Server, stored 

using time zone of the Notes Server. 

The database maintained by the Notes Server holding 

all note related information. 

A transactional history of Notes EvenLs. 



■'^ General Description 

The present invention is a system for coUaborative docu- 
ment annotation whereby notes or annotations associated 
with a document are stored on a web server. Examples of 

35 documents include, but are not limited to, images, text 
documents or documents expressed in a page description 
language such as Postscript or Adobe PDF. In addition, each 
document may contain more than one page, wherein each 
page is annotated independently of the other. The documents 

40 and associated annotations are treated independently from 
each other. Separate data structures are created for the 
documents and for the associated annotations thus permit- 
ting their independent management. The invention can be 
implemented as software, a portion of which executes on the 

45 server side and a portion executes on the client side. The 
server side may comprise a plurality of software applications^ 
running in parallel that in combination provide the server 
functionality of the invention. A _web scrv^| [-, f f,|jnl jpf^ ynn gp ^ 
the server side functions to capture special request from one 

50 or^9l&£lieatuapphcations^wsr6atiflg^,4Q£^ 

retrieying.aqoo^ti QnsjcelaledJLQ-AD^cific dpt^ m ents located / 
iB».^Ss§&tyj^ A notes s eixeiLfuncliQns. to l^g all anaotation l 
activities along' with^infcir malioj^ ^ abo^t J l;ie-CQ rrespondmg 
clients t h aLcpala^ diLa ad-ret rievf,,. them . 

55 Illustrations of an example image displayed with and 
without multiple annotations displayed thereon are shown in 
FIGS. lA, IB and IC. The system of the present invention 
is suitable for use with a variety of document types, includ- 
ing text and image based documents. For illustrative 

60 purposes, the following description uses an image type 
document to illustrate the principles of the invention. This is 
not meant to limit the scope of the invention. One skilled in 
the computer and software arts can apply the system the 
invention to other document types as well without departing 

65 from the spirit and scope of the present invention. 

In FIG. lA, the display 10 is shown comprising a single 
document window 12 in which an image is displayed. No 
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annotations are displayed id this view. In FIG. IB, the 
display 10 is shown comprising a single document window 
14 in which an image is displayed with a plurality of 
annotations 16. The annotations 16 are displayed over the 
image but are not a part of the image itself. In FIG. IC, the 
display 10 is shown comprising two portions: the image 
window 18 and a Note List window 20, The image window 
18 displays the image with the plurality of annotations 16 on 
top of the image. The Note List window 20 displays a list of 
all the annotations associated with the image shown in 
window 18. The user can optionally sort the Note List by 
Note Owner ID, Note Time Stamp, Note color label and 
Note Contents. The display may include Note Event 
description. Note Owner ID, user name, path, originator IP 
address, time and Note Contents. When a note is selected on 
the Note List, its counterpart icon in the document window 
frame is highlighted. 

An illustration of an example annotation is shown in FIG. 

2. The annotation, generally referenced 30, comprises a 
window having a title portion 32 and text portion 34, The 
window contains a note label (Sam), Note Owner ID or login 
name (Acme) and the data and time of the most recent 
synchronization. 

A block diagram illustrating the collaborative document 
annotation system of the present invention is shown in FIG. 

3. The annotation system, generally referenced 40, com- 
prises one or more standard web browsers 42 labeled web 
browser #1 through web browser #N. Each web browser 
comprises a Note Plug-In component 44 that loads into the 
browser. The web browser 42 may comprise any suitable 
browser such as Netscape Navigator or Microsoft Internet 
Explorer, Each web browser in combination with the Note 
Plug-In is referred to as a Notes Client 41 and is the tool used 
for browsing the documents located on the server side. The 
Note Plug-In is invoked each time the browser gets a 
response from the Web Server 54 with a particular Muhi- 
purpose Internet Mail Extension (MIME) identifier. The 
communication protocol between Notes Clients and Notes 
Server is standard Hyper Text Transport Protocol (HTIT). 
The web browsers are adapted to communicate over a global 
TCP/IP network such as the Internet 46. The web browsers 
42 and the Notes Server 58 communicate with the web 
server 54 via the HTTP protocol. 

The server portion of the system 40 comprises a web 
\ server 54 with integrated Note Agent 56, Tsf^^^s Server 58 
'coupled t o a Nojes Database 60, a Document File Server 62 
and a messaging system ^i, a Notes Administrator 50 and a 
Notes Viewer 48. The Notes Server communicates with the 
Notes Database via any suitable database language/protocol 
such as SQL. The Notes Server comniunicates with the 
y^essaging system via standard SMTP. 

Annotation (Note) 

An annotation or note is a portion of text or a graphical 55 
drawing that is associated with a specific location in a 
document. The terms annotation and note are intended to 
mean Ihe same thing and are used interchangeably through- 
out this document. The location associated with a note in the 
document is called a Note Anchor and is kept separate from 60 
the annotation data itself. Once a note is created, its anchor 
point can be changed by the user. The note anchor is 
expressed in terms of (X, Y) coordinates in the annotated 
page of a document. A note anchor may be set to a coordi- 
nate value of (0, 0) if the document associated with the note 65 
is a logical folder. Note that the annotation can be displayed 
within its own window or can be layered on top of the 
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35 



45 



50 
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displayed document. The first option is used in the case of 
text only annotations. The second option is used for mixed 
text and graphical annotations. 

Notes Client 

The application that displays the annotated document and 
allows for annotation sessions is called the Note Client and 
comprises the web browser 42 and Note Plug- In 44. In order ^it 
to differentiate between notes generated by various users,^- ^^'^ 
each user chooses a unique user ID that forms a Note Owner 
identifier. The Not e Owner ID is formed from a combinatio n 
of the chosen user naine and user login name used when 
logging on to the annotation system. The unique Note 
Owner ID is displayed in the note window title bar 32 (FIG, 
2) or when the Note Client application shows note properties 
for a graphical note, i.e., a note not within a window. 

Notes are ass ocia ted with specific documents that are 
storea on tne j^erver S cleJA note can belong to one document 
or to a set of documents aggregated under the same logical 
structure. Documents are document aggregations are iden- 
tlffianSy^ their Note Document ID . The Note Document ID is 
a unique Universa l Resource Locator ^'mjtLI nii the World 
Wide Web. 

It is important to note that the client portion of the 
annotation system of the present invention is not hmited to 
implementation via a web browser and associated plug in. 
The invention can be implemented using any suitable means 
including, but not limited to, stand alone clients that imple- 
ment the same functionality without the use of a web 
browser and/or plug- in software component. 

Notes Server 

The Notes Server 58 functions to provide the centrali 
document and note management features. Normally the/ 
Notes Server functions in conjunction with a web server 54. Y» 
It may access documents that reside in its own file system or / 
that reside on remote file servers 62 located within the local 
area network. In addition, the Notes Server 58 may access 
these file servers and file systems by means of network file 
system protocols (represented by arrow 70). Once accessed, 
their contents are transmitted to the Notes Qient Applica/ 
tions via TCP/IP protocols over communication means such^ ot^*^ 
as LAN, WAN or the Internet 46. Note that the invention I 
may be implemented to operate over an Intranet, e.g., LAN, 7 
or Extranet, e.g., WAN or the Internet. ^ 

An Annotation Session starts when a user logs onto the^ 
Web Server 54 by supplying a user name (user ID) and 
password. The Notes Server 58 requests this informatioi^j 
when the user browsers to a protected URL, which is the I 
URL of a document to be viewed and/or annotated. The | 
Notes Server 58 associates the user name and password with * 
the IP address of the request for identification purposes later 
in the Annotation Session. A session terminates when no 
activity is sensed by the Notes Server 58 for more than a 
predetermined period of time. Note that no special logout 
procedure is required to end an Annotation Session. The 
document protection mechanism can be set in combination 
with the web server security and the document file system 
security. 

The Notes Server assigns a unique Note Serial Nutfiber 
upon the creation of each note. This number serves to 
uniquely identify each note in the system. A Note Event is 
defined as a message related to a note that did not exist at the 
time of the last synchronization with the Notes Server or a 
note that was modified or deleted since that time. Note that, . 
at the client side, the user is permitted to create new note^J 
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modify the contents of notes, modify the anchor points of time a Notes Synchronization is initiated at the client side, 

notes and delete notes. Only users with suflBcient privileges. The time the Notes Client displays to the user for each event 

however, can modify notes. The system can be configured is the Note Time Stamp that was saved in the Notes Server 

such that users without sufficient privileges are not permitted adjusted to the local lime zone of the client, 
to view specific notes or to change them. 5 xhe administrator or other users (i.e. clients) of the Notes 

It is important to note that the server portion of the Server may elect to be notified each time a Note Event is 

annotation system of the present invention is not limited to received by the Notes Server. The notification can be made 

implementation using a web server. The invention can be using any suitable means of notification including, but not 

implemented using any suitable means including, but not limited to visual alarms, audible alarms and/or automatic 

limited to, a stand-alone HTTP server that implements the e-mail messaging sent to the administrator or the user of the 

same functionality without the use of a web server. In system. These notification alerts are termed Note Alarms and 

addition, the invention is not limited to any specific client/ are useful for document management workflow. Alarm noti- 

server protocol implementation, e.g., HTI'P. fications can be generated in accordance with various cri- 
teria including the type of Note Event, type of document. In 
Notes Synchronization is addition, filtering may be performed on a user basis or on a 

Note Events are exchanged between the server and the document basis. 

clients. Notes Clients forward to the Notes Server requests . , _ j xt . * u 

,.r , J 1 , . T *u Notes Server and Notes Database 

to view, modify, create or delete notes. In response, the 

Notes Server Updates the clients with an up to date note list On the Web Server 54, special address aliasing is provided 

per document. This updating activity is termed Note Syn- by means of the Note Agent 56. The Note Agent 56 may be 

chronization. The synchronization process is performed implemented as a Web Server Internet Server Application 

asynchronously and is always initiated by the Notes Client Programming Interface (ISAPI) filter or as a Web Server 

in response to the user, i.e., it is user driven. extension, both techniques being well known in the com- 

Alteraatively, the synchronization process can be initiated puter networking arts. All requests to aUased URLs are 

by the Notes Client in a periodic manner, i.e., polling, or by processed by the Note Agent 56 that in turn transfers the 

the Notes Server when the exchange protocol in use allows requests to the Notes Server 58. 

for it. In the former case, the client would continuously poll The Notes Server 58 may reside in the same computer as 

the server. Utilizing Java in the server, rather than HTTP, the Web Server 54 computer or may reside in another 

permits the server to trigger the client and update the client compu ter loca t ed on the Local Area Network (LAN ), ^^he^ 

display, if permitted to, since it is not practical for the server Notes :yerver Sji j unctions to receive modi lied URLs fromf 

to poll clients. The server pushes new notes to selected the Note Agent 56 and provide in return individual docu 

clients by triggering them to pull new notes from the server. ments and their a ssociated annotations. I f a document holi ' 

This serves to simulate a user pressing a synchronization more than one page, the server may provide one page at a 

button. This service can be provided for those clients that time to the client. N oteJhat the documents mavjce^ic^ e^ iri-the. 

, request it. Note that although the description contained san^e^^lejsvstcm ^^ as ^ the Notes Sery.ec or in anot ber^l£. s&tycr 

herein refers to the user driven synchronization process, the 62 necessed over the local area network . The Notes Server 

scope of the invention is intended to include other processes 58 accesses the document inie server by means of stand 

as well, such as server polling and server initiated synchro- Netwo rk File System (NFS) protocols. 

nization via protocol means. The Notes Server 58 functions to keep track of all N 

If the user elects to exit the display and annotation annotation activity in the Notes Database 60. The Notes ) 

application or chooses to display and annotate another Database 60 can be implemented in numerous ways. A / 

document without synchronization the previous one, the convenient implementation is to use a standard SQL rela- / 

Notes Client appHcation saves the Note Events to the local tional database. A Notes Table (not shown) in the Notes I 
file system. ITie next time the user browses to the same URL 45 Data base 60 is used to reflect the current state of the V 

using the Notes Client application, these events can be annot^ed 'documents. Transaction history is handled by a 7 

synchronized. Thus, the system allows for Note Persistence Notes T^g Table (not shown) also ^iflied-in-the^N&tes / 

pn the client side. Database 60 which iginxi>J-'^a3fiiitexl«as»a-Iransaction log. All 
^ llie Notes Server can be implemented as a process * creation, modification and deletion events are logged on the 
running on the web server computer. Alt ernatively, ii is a 50 Notes Log Table. The database is a useful tool for tracking, 

process runnin fyjin a separate computer that com municates all annotation and other activities on a pe r use or a per 

with the web serve r, iln either case, its functionality is the document basis, in aaaition, a viewer application such as 

same. iTii responsible for processing p ^*^ , ^Y'^nt^i f""^ Note Viewer 48 can optionally be used to generate reports on 

maintaining t he Notes n af^^^<: f.^nd-ib£^ Nntp.s Log. The the histo ry of annotations on a aem&er or a per docume nt 
protocol the Notes Server utilizes is based on the weU- 55 basis that occurred oyer a pcriod^of time, 
known HTTP protocol. The Notes Server generates two EacfTrecord in ihc Note sTable of^the Notes Database 60 

independent HTTP streams: the first stream is for the dis- utilizes the unique 'l^otc '^c'na^ r^TumtecFforlBeT^ index, 

played image and the second stream is for the annotations. Each record in the database comprises the toUowing infor- 

Note again that alternatively, the invention can use any mation: Note Document ID, Note Contents (text or graphical 
TCP/IP based protocol such as Java rather than HTTP. go information). Note Anchor coordinates. Note Time Stamp 

The time of modification (in local client time) is stored as and Note Owner ID. This information is extracted from th^ 

part of the Note Event. The Notes Server ignores the locally Note Event data structure, which is described in more del, * 

generated time stamp and records the time (in server time) hereinbelow. 

of arrival for the event in the Notes Database and the Notes To simplify the implementation of the Notes Database 60, 
Log. ITie time stamp is termed the Note Time Stamp, l^e 65 the Note Contents can be limited to a specific size. For 

Notes Server calculates the difference between its own clock example, the text of a note may be limited to a specific 

and the client clock to obtain the time zone of the client each number of bytes or characters, or the graphic drawing can be 
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limited to a specific number of control points. Creating 
limits on the size of the note allows the record size to be 
predicted in advance. In some systems this may be impor- 
tant. The size limitation can be checked by the Notes Client 
or by the Notes Server when processing the Note Event 5 
transaction. 

The Notes Server may be configured to utilize external 
messaging system 52, such as an electronic mail server, for 
generating alert notifications to specific users. The users 
receiving the alert notifications may or may not be clients of lo 
the annotation system. The alert notifications may comprise 
any portion of the notes themselves along with additional 
information about the annotated document and the severity 
and/or uregency of the message. 

The management of the Notes Server 58 and the Note 
Agent 56 in the Web Server 54 may be performed by a Notes 
Administration tool 50. One of the main functions of the 
Notes Administration tool is to provide the URL aliases 
needed for controlling access to specific documents. 

20 

Security and URL Aliases 

Access permissions to the Notes Server 58 are defined on 
a per client user basis. As part of the task of administering 
the system 40, a directory is created for the user in the 
annotation root folder under the root directory of the web 25 
server 54. 

The Note Agent 56 maintains two translation tables: a 
Document Location Table (Table 1) and a Client Access Key 
Table (Table 2). Both tables are present below. 



TABLE 1 



Document Location Tabic 




Alias Actual Path 




Archive D:\Arc 


TABLE 2 


Client Access Key 'fiible 




Client Name Access Key Location Alias 


Sub Path 


Acme Acmc_Docs Archive 


Acine_filcs 



35 



40 



In order to access the system, a client needs to provide the 
Note Agent Trigger Token as part of the URL. In the 
example presented herein, this is the token 'annotate.* This 
Trigger Token is used by the Web Server 54, e.g., Microsoft jq 
Internet information Server (IIS), to invoke the filter in the 
Note Agent 56 to translate the document path specified in the 
remainder of the URL. The requested URL is as follows 
below. 

Requested URL=http//domain_name/annotate/Acme__ 55 
Docs/Subfolder/document.name where 
domain name='domain_narae* 

Note Agent Trigger token=' annotate *pl Customer Access 
key = ' Acme_Docs* 
In accordance with the above described aliaisng mechanism, 60 
the system 40 may ask users to type in their unique user 
name and password in order to obtain permission to browse 
particular locations or file systems in the local area network 
on the server side. In this manner, users can access their 
private documents for display and annotation purposes while 65 
not violating the privacy and security of other users of the 
system. 
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For example, assuming a Microsoft IIS Web Server 54 
that defines 'C:\Inetpub\wwwroot* as the base document 
root, the NOtes Server 58 will utilize a sub folder called for 
example, 'annotate.' This specific name 'annotate* would 
trigger the Note Agent ISAPI filler 56 in the Web Server 54 
to translate the URL as specified below. 

If *Acme' is a name of a user of the system, a folder 
named *Acme_Docs' will be created under the 
'C:\Inetpub\wwwroot' directory. This folder is the Client 
Protocol Folder. Using the operating system security flags, 
only a user named 'Acme* would be given rights to browse 
this folder. 



Client Protected Folder-' C\Inetpub\wwwroot\annotate\Acme' 

The Note Agent 56 functions to provide mechanisms for 
translating URL addresses under the ' Acme_Docs' folder in 
such a way that logical names can point to physical folders 
on the same computer or on other computers located in the 
network. In connection with the example presented above, 
the Client Document Root physical path is as follows. 

Client Document Root physical path-* d:\ArcVAcme_flles' 

The Client Document Root physical path is translated to the 
following URL. 

Translated UTL_'http;//domaiii_namc/annotatc/Acmc_DQCs* 

As a result, the 'Acme* client is granted permission to 
access any document under her/his Document Root folder. 
Note that neither the Document Location Table (Table 1) nor 
the Document Access Key Table (Table 2) is limited in 
length. Thus, a client may have more than one access key. In 
addition, there may be more than one alias associated with 
the same physical folder; each alias providing protection for 
a different client. 

Annotation Session: Initialization 

The following describes the initialization portion of an 
Annotation Session in more detail. A flow diagram illustrat- 
ing the initialization portion of a general Annotation Session 
is shown in FIG. 4. Note that the client/server protocol used 
throughout the session is the H ITP protocol. 

The Session starts when the user launches a web browser 
and provides a URL in the form that was described above 
and in FIG. 3. This URL refers to a directory or to a specific 
document. The document may contain one or more pages. 
An example. for a single page document is a TIFF file format 
image. An example of a multi-page document is an Adobe 
Portable Document Format (PDF) document. 

Initially, the user launches her/his web browser (step 80). 
To initiate the initial request, the client, via her/his web 
browser, provides a URL (step 82) which is transmitted to 
the server (step 84). The server verifies whether the client 
was already authenticated (step 86). If the client was not 
previously authenticated, the server requests authentication 
from the user in response (step 88). To authenticate a user, 
the browser first displays a dialog to the user, asking for a 
user name and password. The browser then transmits the 
user-supplied information to the server (step 90). The web 
server then checks the credentials submitted against those 
defined for the requested URL (step 92). llie construction of 
the URL and associated aliases was described previously in 
the section entitled Security and URL Aliases. 
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If the user name and password provided by the user are from the server (step 124). The URL that is supplied to the 

not correct (step 94), an error message is displayed (step 98) server to retrieve the notes is the same UL corresponds to the 

and the authentication portion is repeated (step 88). The user underlying document (image) but having a suflBx of '\notes* 

is then queried again for the correct user name and pass- appended to it. Upon receiving the appended URL, the 
word. If the user name and password provided by the user 5 server searches the Notes Database 60 (FIG. 3) for the 

are correct (step 94), the server responds with the directory specified document using the URL as the key for searching 

listing of the URL requested by the user or with the contents (step 126). Once found, the server filters the notes so as to 

of a document if the URL was referring to a specific supply only those notes that the requesting client has per- 

document (step 96). mission to view (step 128). The server subsequently serial- 

The client then displays the directory listing in a window izes the notes after filtering and stores them in a response 

within the web browser (step 98). It is likely that the buffer (step 130). The notes buffer is then sent as a response 

directory contains documents that the user may wish to to the client (step 132). The response buffer data structure is 

annotate at some point in time. Note that the list of docu- described in more detail hereinbelow. 

ments sent by the server when browsing is first filtered by Upon receipt of the notes buffer, the client updates its 
the Notes Server 58 (FIG. 3) to display only those docu- ^5 locally stored Notes Database (step 134) and then displays 

ments with file types that the Note Server knows how to the contents of the notes buffer in a window within the web 

interpret. browser (step 136). If the user has selected to view the 

A message flow diagram illustrating the transaction annotations overlying the document, i.e., the image, then the 

sequence between chent and server during initialization is client displays the notes similarly to that shown in FIG, IB. 

shown in FIG. 5. As described above, the first request The user then can process one or more notes, i.e., create new 

(referenced 100) is made by the client with the requested notes, edit existing ones and/or delete one or more notes 

URL to be displayed. The server first verifies the authenti- (si^p 138). 

cation of the user. If not user name or password was The client is now ready to display notes and accept 

submitted, the server responds with an authentication modifications to them. The Notes Server has supplied the 

request (referenced 102). Once received by the client, the number of notes in the Server Annotation Response Data 

user is queried for a user name and password. The user- Response Structure (FIG. 12). If the number of notes is not 

supplied input is sent as another request to the server equal to zero, the cHent checks the state of the viewing 

(referenced 104). The server attempts to verify the authen- condition flag in the web browser Note Plug-In application, 

tication and if it passes, the server first filters and then sends This variable may have the following values; show docu- 

the directory fisting associated with the requested URL to ment only (corresponding to FIG. lA), show document in 

the client (referenced 106). Upon receiving the directory addition to notes (corresponding to FIG. IB), show docu- 

listing, the chent displays its contents in window in the web ment and notes in one web browser frame and a list of notes 

browser. in another web browser frame (corresponding to FIG. IC). 

. • o • n • ->c case the third option is selected, another web browser 

Annotauon Session: Browsing 35 ^^^^^ ^^^^^^^ document is redrawn in the right 

The following describes the browsing portion of an Anno- frame and the list of notes is drawn in the left frame. The 

tation Session in more detail. A flow diagram illustrating the notes displayed in the list in the left frame can be sorted and 

document browsing portion of a general Annotation Session filtered using criteria such as, for example, time of creation, 

is shown in FIG. 6. Note that the client/server protocol used Note Owner ID, Note Contents, noted shape type, and note 

throughout the session is the HTTP protocol. color. 

The chent first issues a request to the server to view a In accordance with the invention, the client is capable of 

particular document by selecting its corresponding docu- displaying the notes as they arrive from the server regardless 

ment URL (step 110). Before the server responds with the of the state of the document image stream display. This 
requested data, it needs to validate the user's credentials 45 allows the user to begin manipulating the annotations when 

once again since this request to view a document is consid- large image documents are still in the midst of being 

ered a new connection to the sever (step 112). Assuming the transmitting from the server. 

user's name and password were supplied previously, the web a message flow diagram illustrating the transaction 

browser provides the information automatically, thus avoid- sequence between client and server during a document 
ing an error message response from the server. If the user is jq browsing general Annotation Session is shown in FIG. 7. As 

not authenticated (step 114), an error message is returned described previously, the client initiates a document brows- 

and displayed on the client browser (step 116). The docu- ing session by sending the document URL request to the 

ment is not transmitted to the user until the user is authen- server (reference 140). The server then validates the user 

ticated. The mechanics of user name and password authen- permissions including user name and password. Once the 
tication by the server was described in detail previously in 55 user is authenticated, the server responds to the client with 

the Initialization section. the document MIME type and the corresponding document 

Once authenticated, the server then responds by sending data (image data in this example) (referenced 142). 

the document data, i.e., image data in this example, to the The client checks the MIME type and, in response, loads 

client with the proper MIME type (step 118). The MIME the Note Plug-In into the web browser. Once loaded, the 
type sent to the client in response to the request is unique and eo image is displayed in a window in the browser. Then, the 

is the name for all document types. In response to receiving annotations associated with the image are requested from the 

the MIME type, the web browser at the client, loads the Note server by sending the URL corresponding to the document 

Plug-In (44 (FIG. 3) on the client (step 120). The Note with the test Vnotes' appended to it (referenced 144). The 

Plug-In client functions to take control of the browser server searches the Notes Database and filters the notes for 
display window and begin displaying the image (step 122). 65 those corresponding to the image that the user has permis- 

Immedialely thereafter, the client issues a request to sions for. 'Vhc resulting groups of notes are serialized and 

retrieve the notes associated with the image, if there are any, placed in a notes buffer. The buffer is then transmitted to the 
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client (referenced 146). The client receives the buffer. Notes Client, in response to the button press, functions to 

updates the local note database, displays the notes in a prepare a note buffer to send to the Notes Server (step 152). 

window in the browser and gets updates to one or more notes The note buffer contains only the notes that were changed 

from the user. since the time of the previous synchronization event. The 
In accordance with the invention, the Note Plug-In run- 5 time of the prcvioiis synchronization event b taken as cither 

ning on the client web browser provides the user with the "'f 'fj""' ^f^"^^' '° '^^''^"^ '="'7f °' f'^''; 

u i\. ^^„^,„i T« me nt or the actual time the user previously pressed the sync 

capab.luy to pe form several operations on notes. In ^^^^^^ ^ ^^.^^^ application. The time of the last 

particular, the notes client permits a user to perform the synchronization event is always known and stored by the 

tollowing. Notes Qient application. This time is also transmitted by the 

Notes can be created by selectitig an annotation tool on Notes Server in the Server Annotation Response Data Struc- 

the screen and placing the cursor in the area of the image ture. 

where it is desired to place a note. A box is created and the xhe client then serializes the notes within the note buffer 

user can enter text. Once entered, the text appears in the (step 154). After serialization of the note data, the client 

form of a 'slick' note that appears on top of the image. The users the data in the buffer to construct a Qient Annotation 

Note Anchor is created at the location the user placed the Event Data Structure (FIG, 10) and£QS34^to the Notes 

note. Notes can be viewed by double clicking, for example, Server using the original URL provided by the user to 

on a note icon. Note icons are displayed on the screen when retrieve the notes previously (step 156). The server validates 

the viewing option is set to display both notes and the image. the user name and password (step 158) and merges the note 

A note can be closed after viewing by clicking on the box in event information with the Notes Database 60 (FIG. 3) (step 
the upper left-hand corner of the title bar 32 (FIG. 2). Note 20 160). The merge process includes checking whether or not 
that alternatively, graphics can be entered into a note rather has the required permission level to modify notes, 

than test. In this case, a graphics tool or marker is used to Those notes, i.e. Note Events, that the user is not authorized 

place graphics in the note. ^^^''^Tu ''^ u^L"''^c°°^iVt'' ^ ""Vt^' 

^ . , ,.. ... mdicated by a zero in the Note Serial Number field, are 

Anote can be moved to a new location by grabbing a note ^^-g^^^ ^ ^^-^^^ ^^^^ ^^^-^^ mmbtr, A new record is 

icon and draggmg it to a new location on the image. The ^^.^^^^j ^^^^^ Database and th^^^ nn,. pj^^H jp 

Note Anchor is changed to correspond to the new location. j^e recor d (steaJ 62). A new recordTTalso created in the 

Other features include printing the document with the notes NiJmr'lUblelTfe transaction is also recorded in the Notes 

attached; scrolling to another section of the document with Log Table (step 164). 

the notes scrolling with the document; zooming in or out Next, the Notes Server prepares a Server Annotalion 
within the displayed document; deleting a note; resizing a Response Data Structure filled with the new list of notes and 
note that contains either test or a graphic drawing; changing their content (step 166). An important aspect of the invention 
the default note color, dimensions and associated default is that the list of notes includes changes made by all 
. marker. Further, notes can be synchronized with the Notes concurrently active clients that have previously synchro- 
Server, which is described in more detail hereinbelow. nized their notes with the Notes Server. In this fashion, the 
It is important to note that a user may create as many notes annotations associated with a particular document made by 
as she/he desires. Existing notes can be modified even if ^11 other clients are replicated to each client at the time they 
other usere created them. The server, however, will not perform note synchronization. 

permit any changes that the user is not authorized to make. The Notes Server responds to the synchromzing cHent by 

In accordance with the invention, all changes made to notes transmitting the Sender Annotation Response Data Structure 
are saved in a local note database on the client. The database ^^e chent (step 168). Once received the cUent merges the 

is managed within the RAM of the computer and is backed !^°f ^"^'Y'^'l local notes database (step 170). The display 

^ 1 , , ^ J • • u J J- 1 IS then refreshed and the Notes Client is ready to receive and 

up on the local computer mass storage device. I.e., hard disk ■ *u • / * i'7'i\ 

^K^^ ^ & » » process input from the user once again (step 172). 

' - . . A message flow diagram illustrating the transaction- 

This backup of the database also functions to provide a 45 ^ce between cUent and server during note synchroni- 

recovery mechanism m case the user exits the application or ^^^.^^ ^ ^^^^^ 9 ^ described previously, the 

requests a different URL without synchronizmg with the synchronization process is initiated and driven by the user, 
server beforehand. The database is kept for a limited period -^^-^^^^ ^ ^^^^ ^^^^j ^ ^^^^3 

of time before being erased TTie time penod is set by the ^-^^ ^^^^^ processed by the Notes Server and - 

user IS typically on the order of a few days. Whde the 50 p^s^s. i.e., transmits, the buffer to the server (referenced! 

database is saved, the user retains the option of synchroniz- ^g^^ ^^^^^ ^^^^ ^ ^j-^^^ Annotation 

ing any unsaved notes the next time she/he browses to the ^^^^^ j^^^^ Structure. The Notes Server, in response, update! 
same document URL. ^^^^^ Database after authenticating the user. Any actions I 

The structure of the local note client database is similar to requested by the user that are not authorized due to lack ol} 
that of the server note database with the exception that no 55 permissions, are ignored. 

authentication checks are performed on the client. Thus, for ^^^^^^^ processed by the server and a response 

example, the note database on the client does not contain a buffer is created. The buffer is sent to the client as a Server 

Note Owner password field. • Annotation Response Data Structure (referenced 182). The 

Annotation Session: Synchronization ^"^0^, upon receipt of the notes list, updates its local notes 

database, displays the notes in a frame in the web browser 
The following describes the synchronization portion of an [^^^^ ^r more notes commands from the user. The 

Annotation Session in more detail. A flow diagram illustrat- process is repeated while the user continues to create, edit 

ing the note synchronization portion of a general Annotation ^nd delete notes. 
Session is shown in FIG. 8. The process of synchronizing 

notes is initiated and driven by the user. ITie user initiates the 65 ^1^^°* Annotation Event Data Structure 

synchronization process, for example, by pressing a button A diagram illustrating the Client Annotation Event Data 

in the browser application called *Sync' (step 150) The Structure in more detail is shown in FIG. lO.fcT^ff^Jata 
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Structure, generally referenced 190, comprises a plurality of 
fields for conveying note data from the Notes Client to the 
Notes Server. The data structure 190 comprises a Document 
ID field 192 which comprises the URL of the annotated 
document the notes are associated with, local time 191 of the 
client along with the time zone of the client, the Note Owner 
ID 196 which uniquely identifies the user and the number of 
notes 198 contained in the message. Following the number 
of notes is a buffer containing one or more notes labeled note 
#1 ^tog^gh note #N 202. 

""""^^Hiagram illustrating the note data structure of the Client 
Annotation Event Data Structure of FIG. 10 in more detail 
is shown in FIG. 1. The data structure, generally referenced 
' 21p, c omprises a pluraltty oi neids that are used by tne client 
)to repi'esent each note . Each note comprises^aiNoie i»eriai 
P^TTHb^rSlSTieid uniquely identifying the note. The Note 
ferial number is set to zero for a new note and set to a 
non-zero value for an existing note. An action field 214 
represents the action to be taken on the note. The following 
values are valid actions. 



0 


No action 


1 


Add a note 


2 


Modify a note 


3 


Delete a note 


4 


Add a comment 


5 


Update status 
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A note Anchor field 216 stores the coordinates of the 
position of the note in the document. The note status field 
218 can have the following values. 



Ignore 

To be performed 
Done 



35 



The note color field 220 represents the color the note is to be 
displayed in. The note color represents a RGB value. 

The text length of the note 222 represents the number of 
bytes or characters of the next note itself. The note test field 
224 represents a buffer with the actual text of the note. The 
note shape field 226 can have the following values. 



45 



0 


No shape 


1 


Polygon 


2 


Spline 


3 


Rcctange 


4 


Ellipse 


5 


Marker 



A note shape line cap field 228 can have the following 
values. 
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0 


No marker 


1 


Star 


2 


Circle 


3 


'Approved' sign 


4 


'Rejected* sign 


5 


'Correa' sign 


6 


'Remove* sign 



10 



The number of shape vertices field 234 represents the 
number of points in the shape and the shape vertices 236 
comprise a plurality of N (x, y) coordinates. 

Server Annotation Response Data Structure 

A diagram illustrating the Server Annotation Response 
Data Structure in more detail is shown in FIG. 
structure, generally referenced 2 40^_comp ris£s a pj[|^j;-^ iuy ot] 
fields for conveying the no te-datajrom tl^ g^ ^otes Server td 



the N^fef;,,f;i^erit. The data Structure 240 comprises a Docu j 
ment ID field 242 which comprises the URLof t hej 

docum enjjh e^^notes are as&o ciaiea wit EJ^loffnl Gfl^ 

244, thT'^imensions of ^ the "'Hocume rllr including the 
width ari&"*tRflgBt'"iTrimIlimetel^^ of the 

width and height in pixels per milli meter_an d,the-min]brtf;i)f 



30 



notes 248 contained in the me^aggf . Following the number 
of notes in a buffer containing^one or more notes labeled 
note #1 250 through not #N 252. 

A diagram illustrating the note data structure of the Server 
Annotation Response Data Structure of FIG. 12 in more 
detail is shown in FIG. 13. llie daU ^tructui^^ generallv^^ 
referenced 260, comprises a plurality nf q^)f) s fih^'.fTi .M^^rV^ 
by the server to represent each no.t e. Each note comprises a 
Note Serial Number 262~field uniquely identifying the note. 
The Note Serial number is set to zero for a new note and set 
to a non-zero value for an existing note. The note Time 
Stamp field 264 represents the time (in server time) of the 
update for this particular note. An action field 266 represents 
the action to be taken on the note. The following values are 
valid actions. 



0 


No action 


1 


Add a note 


2 


Modify a note 


3 


Delete a note 


4 


Add a comment 


5 


Update status 



A Note Anchor field 268 stores the coordinates of the 
position of the note in the document. The note status field 
270 can have the following values. 



55 



0 None 

1 Right arrow 

2 Left arrow 

3 Both arrows 

4 Round 

5 Square 



The note shape line width field 230 represents the width of 
the shape line in millimeters. The note marker type field 232 65 
represents the type of marker and can have the following 
values. 



0 Ignore 

1 To be performed 

2 Done 



Tlie note color field 272 represents the color the note is to be 
displayed in. The note color represents a RGB value. 

The test length of the note 274 represents the number of 
bytes or characters of the text note itself. The note test field 
276 represents a buffer with the actual text of the note. The 
note shape field 278 can have the following values. 
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0 


No shape 


1 


Polygon 


2 


Spline 


3 


Rectangle 


4 


Ellipse 


5 


Marker 


A note shape line cap 


field 280 can have the following 


values. 




0 


None 


1 


Right arrow 


2 


Left arrow 


3 


Both arrows 


4 


Round 


5 


Square 
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database and to return to said one of said notes clients 
an updated notes list incorporating annotations previ- 
ously made by all other notes clients that have syn- 
chronized with said notes server since synchronization 
5 between said one of said notes clients and said notes 
server was last performed. 
2. The system according to claim 1, wherein said notes 
database comprises a Notes Table for storing the current 
content of a note. 
10 3. The system according to claim 1, wherein said notes 
database comprises a Notes Log Table for storing a trans- 
action history of all note events for notes associated with a 
particular document. 

4. The system according to claim 1, said notes client 
15 comprises: 
a display; 

network communication means; 
a processor suitably programmed to run; 
a web browser; and 

The note shape line width field 282 represents the width of ^ client note plug-in adapted to run within said web 

the shape line in millimeters. The note marker type field 284 browser. 

represents the type of marker and can have the following 5. The system according to claim 1, wherein said network 
values. comprises the Internet. 

25 6. The system according to claim 1, wherein said network 
comprises a TCP/IP based network. 

7. llie system according to claim 1, wherein said network 
comprises an Intranet. 

8. The system according to claim 1, wherein said notes 
server comprises; 

network communication means; 
a process suitably programmed to: 
provide web server services; 
^ . ^ , .^.,>,„^ . provide notes agent services; and 

The number of shape vertices field 286 represents the 35 -^^ document and notes browsing and synchroni- 

number of points m the shape and the shape vertices 288 ^^-^^ ^^-^^ ^^^^ ^U^^^ 

comprise a plurahty of N (X, Y) coordinates. , j^^^ ^ ^^^^j g ^^^^^j^ 

While the mvention has been described with respect to a „„^„* „^ « ♦ c a t- 

. , , r , 1. -n . . 1 , agent services comprises an Internet Server Appucation 

hmited number of embodiments, it will be appreciated that programming Interface (ISAPI). 

many variations modifications and other applications of the ^^^^^^ according to claim 1, further comprising 

invention may be made. ^ ^^^^^ administrator for providing one or more Universal 

What IS claimed iS; Resource Location (URL) aliases in managing and admin- 

1. A system for collaborative document annotation, com- 

. . ^ * istratmg said notes server. 

pnsing. system according to claim 1, further comprising 

a document file and a notes database file located on one 45 ^ note viewer for generating reports on the history of 

or more servers, said document file for storing one or annotations on a per user or a per document basis that have 

more documents, said notes database for storing one or occurred over a period of time. 

more notes, each note associated with a particular 12. The system according to claim 1, wherein two or more 

document; users can collaboratively work on the same document 

one or more notes clients coupled to a network, each notes 50 whereby each user is capable of viewing annotations made 

client operative to locally display a representation of a by the other users by synchronizing with said notes server, 

document remotely stored in said document file, each 13. ^[^q system according to claim 1, wherein a single 

notes client adapted to enable a user to annotate said page of a multipage document can be annotated on a notes 

local version of .said document with one or more notes, client without requiring the entire multipage document to be 

wherein any number of users in said network may 55 transmitted to said notes client. 

simultaneously annotate their respective said local ver- 14. jhe system according to claim 1, further comprising 

sion of said document; security means for providing Universal Resource Locator 

a notes server of said one or more servers being coupled (URL) translation and redirection services so as to provide 

to said network and operative to manage and maintain user level security for documents and notes stored on said 

said notes database and to process and respond to notes go server, 

events generated by said one or more notes clients; and 15. The system according to claim 14, wherein said 

synchronization means within each notes client and said security means comprises a Document Location Table and a 

notes server, wherein in response to a synchronization Client Access Key Table. 

event initiated by one of said notes clients said syn- 16. The system according to claim 1, wherein said docu- 

chronization means adapted to send notes events gen- 65 ment comprises an image. 

erated by said one of said notes clients to said notes 17. The system according to claim 1, wherein said docu- 

server for merging with the contents of said notes ment is represented using a page description language. 



0 


No marker 


1 


Star 


2 


Circle 


3 


'Approved' sign 


4 


'Rejected' sign 


5 


'Correct' sign 


6 


'Remove' sign 
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18. The system according to claim 1, wherein said notes 27. The method according to claim 22, further comprising 
client is adapted to permit said user to annotate said docu- the step of sending a fourth response from said server in 
ment in parallel with the display of said document. response to receiving a request comprising a folder URL, 

19. The system according to claim 1, wherein said docu- said fourth response comprising a directory listing of said 
mcni comprises a plurality of pages whereby each page can 5 folder. 

be annotated independently of all the other pages. 28. The method according to claim 22, wherein said 

20. The system according to claim 1, wherein said syn- document comprises an image. 

chronization means is user initiated. ^ 29. The method according to claim 22, wherein said 

21. Tlie system according to claim 1, wherein said syn- ^^^T^* represented usmg. a page description lan^age. 
chronization means is triggered by said notes server via the lo The method accordmg to claim 22 wherem said step 

t £ 1 , t ^^ / 01 permitting said user to perform annotation functions on 

signaling of selected notes clients. ^^^^ ^^^^ simultaneously with the display of 

22. A method of coUaborative document annotation, said document 

method comprising the steps of: ' 3^ ^^^^od according to claim 22, wherein said 

receiving a first request comprising a Universal Resources document comprises a plurality of pages whereby each page 

Locator (URL) associated with a document to be 15 is capable of being annotated independently of all the other 

viewed and annotated and transmitting said first request pages. 

from a client to a server; 32. The method according to claim 22, wherein said steps 

sending a first response from said server to said client in of gerierating a third request and receiving a third response 

response to said first request, said first response com- ^ initiated by said user. 

prising a document type and a representation of said The method according to claim 22, wherein said steps 

requested document; °f generating a third request and receivmg a third response 

, , - A . . is initiated by said server via the signaling of each selected 

receiving a second request comprising a document note . ^ & & 

URL corresponding to the notes associated with said j- ,-<-,,. 

document and sending said second request from said , ^^'.^he system accordmg to claim 17, wherem said page 

client to said server* ^5 description language is portable document format or PDF. 

' ^ . , . , 35, The method according to claim 29, wherein said page 

sending a second response from said server lo said client description language is portable document format or PDF. 

m response to said second request, said second 3^ y^^^ ^ according lo claim 1, wherein said syn- 

response comprising one or more notes associated with chronization means within said notes server, in response to 

said document, ^ synchronization event involving one of said notes clients, 

displaying the representation of said document and the alerts others of said notes clients of said synchronization 

associated one or more notes locally on said client such event, 

that both are viewable simultaneously; 37^ xhe system according to claim 36, wherein said 

providing said user annotation editing functions on said synchronization means within said notes server alerts said 

client wherein any number of users in said network 35 others of said notes clients by initiating transmission of an 

may simultaneously annotate their respective represen- electronic maD or e-mail message to said other of said notes 

tations of said document and, in response to a synchro- clients, 

nization event initiated by said client, generating a third 38. The system according to claim 36, wherein said 

request comprising one or more note events related to synchronization means alerts only notes clients that have 

said document subsequently transmitted from said cli- 40 elected to be alerted of synchronization events and does not 

ent to said server; and alert notes clients that have not elected to be alerted of 

sending a third response from said server to said client in synchronization events. 

response to said third request, said third response 39, The method according to claim 22, further comprising 

comprising an updated list of notes associated with said the step of said server, in response to said third request, 

document, said updated list of notes comprising the 45 alerting chents other than said client fi-om which said third 

results of a merge between the note events sent in said request was received that said note event was transmitted 

third request and a notes database in communication from said client to said server. 

with said server, said updated list of notes incorporating 40. The method according to claim 39, wherein said 

annotations previously made by all other clients that server alerts said clients by initiating transmission of an 

have synchronized with said server since synchroniza- 50 electronic mail or e-mail message. 

tion between said client and said notes server was last 41, The method according to claim 39, wherein said 

performed. server alerts means alerts only clients that have elected to be 

23. The method according to claim 22, wherein said alerted of synchronization events and does not alert clients 
document type is a predetermined type regardless of what that have not elected to be alerted of synchronization events, 
type of medium said document represents. 55 42. The system according to claim 1, wherein each notes 

24. The method according to claim 22, wherein said step client receives from said notes server an indication of a type 
of getting a first request comprises the step of querying said associated with said document, said type being a predeter- 
user for a user name and password. mined type regardless of what type of medium said docu- 

25. The method according to claim 22, wherein said step ment represents. 

of sending a first response comprises the step of authenti- 60 43. The system according to claim 42, wherein said 

eating said user on said server before sending said first indication of a type is a Multipurpose Internet Mail exten- 

response. sion (MIME). 

26. The method according to claim 22, further comprising 44. The method according to claim 23, wherein said type 
the step of getting a fourth request comprising a folder URL is indicated by a Multipurpose Internet Mail Extension 
associated with the folder comprising one or more docu- 65 (MIME). 

ments to be viewed and annotated and transmitting said 45. A system for collaborative document annotation, 

fourth request from said client to said server, comprising: 
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one or more servers, coupled to a first network; 

a document file system located on at least one of said 
servers, said document file system for storing document 
files, said document files describing document pages; 

a notes database located on at least one of said servers, 
said notes database for storing one or more notes, each 
note associated with a particular one of said document 
pages; 

one or more notes clients coupled to a second network, 
each notes client operative lo locally generate a display 
of a document page, described by one of said document 
files, said notes client adapted to permit a user to 
annotate said document page with one or more user 
notes, said notes client operative to generate note 
events describing user notes, wherein any number of 
users in said network may simultaneously annotate 
their respective said document page; 

a notes server of said one more servers being coupled to 
said second network, said notes server operative to 
receive one or more note events processed by one of 
said notes clients; and 

synchronization means within each notes client and said 
notes server, wherein in response to a synchronization 
event initiated by one of said notes clients said syn- 
chronization means adapted to update said notes server 
with any notes events processed by said one of said 
notes clients and to return to said one of said notes 
clients an updated notes list incorporating annotations 
previously made by all other notes clients that have 
synchronized with said notes server since synchroni- 
zation between said one of said notes clients and said 
notes server was last performed. 

46. The system according to claim 45, wherein said notes 
database comprises a Notes Table for storing the current 
content of an archived note. 

47. The system according lo claim 45, wherein said notes 
database comprises a Notes Log Table for storing a trans- 
action history of all note events for notes associated with a 
particular document file. 

48. The system according to claim 45, wherein said notes 
client comprises: 

a display; 

network communication means; 

a processor suitably programmed to run; 

a web browser; and 

a client note plug-in adapted to run within said web 
browser. 

49. The system according to claim 45, wherein said 
second network comprises the Internet, (previously added) 
(New) The system according to claim 45, wherein said 
second network comprises a TCP/IP based network. 

50. The system according to claim 45, wherein said 
second network comprises a TCP/IP based network. 

51. The system according to claim 45, wherein said 
second network comprises an Intranet. 

52. The system according to claim 45, wherein said notes 
server comprises: 

network communication means; 
a processor suitably programmed to: 
provide web server services; 
provide notes agent services; 

provide document and notes browsing and synchroniza- 
tion services to said one or more notes clients. 

53. The system according lo claim 52, wherein said notes 
agent services comprises an Internet Server Application 
Programming Interface (ISAPl). 
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54. The system according to claim 45, further comprising 
a notes administrator for providing one or more Universal 
Resource Location (URL) aliases in managing and admin- 
istrating said notes server. 
5 55. The system according to claim 45, further comprising 
a note viewer for generating reports on the history of 
annotations on a per user or a per document file basis that 
have occurred over a period of time. 

56. The system according to claim 45, wherein two or 
10 more users can collaboratively work on the same document 

file whereby each user is capable of viewing notes made by 
the other uses upon synchronization with said notes server. 

57. The system according to claim 45, wherein a docu- 
ment page can be annotated on a notes client without 

15 requiring the entire document file to be transmitted to said 
notes client. 

58. The system according to claim 45, further comprising 
security means for providing Universal Resource Locator 
(URL) translation and redirection services so as to provide 

20 user level security for documents and notes stored on said 
servers. 

59. The system according to claim 58, wherein said 
security means comprises a Document Location Table and a 
Client Access Key Table. 

25 60. The system according to claim 45, wherein said 
document page comprises an image. 

61. The system according to claim 45, wherein said 
document page is represented in said document file using a 
page description language. 

30 62. The system according to claim 45, wherein said notes 
client is adapted to permit said user to annotate said docu- 
ment page in parallel with the display of said document 
page. 

63. The system according to claim 45, wherein said 
35 document file comprises a plurality of document pages, 

whereby each document page is able to be annotated inde- 
pendently of all the other document pages. 

64. The system according to claim 45, wherein said 
synchronization means is user initiated. 

40 65. The system according to claim 45, wherein said 
synchronization means is triggered by said notes server via 
signahng of selected notes clients. 

66. The system according to claim 61, wherein said page 
description language is portable document formal or PDF. 

45 67. The system according to claim 45, wherein said 
synchronization means within said notes server, in response 
to a synchronization event involving one of said notes 
clients, alerts others of said notes clients of said synchroni- 
zation event. 

50 68. The system according to claim 67, wherein said 
synchronization means within said notes server alerts said 
others of said notes clients by initiating transmission of an 
electronic mail or e-mail message to said others of said notes 
clients. 

55 69. The system according to claim 67, wherein said 
synchronization means alerts only notes clients that have 
elected to be alerted of synchronization events and does not 
alert notes clients that have not elected to be alerted of 
synchronization events. 

60 70. The system according to claim 45, wherein each notes 
client receives from said notes an indication of a type 
associated with said document file, said type being a pre- 
determined type regardless of what type of medium said 
document file represents. 

65 71. The system according to claim 70, wherein said 
indication of a type is a Multipurpose Internet Mail Exten- 
sion (MIME). 
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72. The system according to claim 54, wherein said provide notes agent services; 

Universal Resource Location (URL) aliases are composed of provide document and notes browsing and synchroniza- 

document file location within said document file system. tion services to said one or more notes clients. 

73. A system for collaborative document annotation, 79. The system according to claim 73, further comprising 
comprismg: 5 a notes administrator for providing one or more Universal 

one or more servers coupled to a first network; Resource Location (URL) aliases in managing and admin- 

a document file system located on at least one of said istrating said notes server, 

servers, said document file system for storing document 80. The system according to claim 73, wherein two or 

files, each stored document file describing one or more more users can collaboratively work on the same document 

document pages in a format defined by a native type file whereby each user is capable of viewing notes made by 

associated with the document file; the other users upon synchronization with said notes server. 

a notes database located on at least one of said servers, ^l- system according to claim 73, wherein a docu- 

said notes database for storing one or more archived "^^"^ P^g^ ^an be annotated on a notes client without 

notes, each archived note associated with a particular requiring the entire document file to be transmitted to said 

one of said document pages; "^^^^ client. 

one or more notes clients coupled to a second network, The system according to claim 73 fiirther comprising 

each notes cUent operative to locally generate a display ^""''[y ^, P^^ovidrng Universal Resource Locator 

of a document page described by one of said document (^^^ ? redirection services so as to provide 

files, said notes client adapted to permit a user to ^^^^^ ^^^^^'^ documents and notes stored on said 

annotate said document page with one or more suer * 

notes, said notes chent operative to simultaneously ^ ^y^^^"^ accordmg to claim 73, wherein said 

display said archived nots associated with said docu- "^""IT^^ ^^^^ comprises an miage 

ment page and said user notes over said display of a ^ The system accordmg to claim 73, wherem said 

document page such that said document page is view- ^^^"^^"^ P^S^ is represented m said document file usmg a 

able along with both said assc)ciated archived notes and P^f/if^"P^'°"l^^^^ , . u • 

said user notes, wherein any number of usen; in said The system accordmg to claim 73, wherem said notes 

network may simultaneously annotate their respective ^^^^^ to permit said user to annotate said docu- 

said document pages; and P^g^ P^^^"^* ^^^^ ^^P^^y ^^^^ document 

a notes ^rver of said one or more servers being coupled 30 ^'f^ ^ ^^^^j- ^^^-^ ^4 ^^^^^^^ ^^-^ 

to said second network and operative to maintain said description language is portable document format or PDF. 

notes database, and ^^^^^^ according to claim 73, wherein said 

synchronization means operative to send one or more indication of a type is a Multipurpose Internet Mail Exten- 

notes events from a synchronizing notes clients to said gj^^ (MIME). 

notes server, merge said notes events with the notes 35 gg. The system according to claim 79, wherein said 

stored in said notes database associated with a particu- Universal Resource Location (URL) aliases are composed of 

lar document, and to send an updated notes list to said document file location within said document file system, 

synchronizing notes cUent, wherein said updated notes 39 -j^e system according to claim 1. wherein said notes 

list IS generated by mergmg said one or more notes ^,1^^^,^^ perform synchronization events with said notes 
event with notes stored in said notes database associ- 40 server a synchronously with respect to each other, 

ated with said particular document, and comprises 90. The system according to claim 45, wherein said notes 

annotations previously made by all other notes clients clients perform synchronization events with said notes 

that have synchronized with said notes server since server asynchronously with respect to each other, 

synchronization between said one of said notes clients 9^ The system according to claim 73, wherein said notes 
and said notes server was last performed. 45 clients perform synchronization events with said notes 

74. The system according to claim 73, wherein said notes server asynchronously with respect to each other. 

client comprises: 92. A method of collaborative document annotation in a 

a display; system including one or more notes clients and a notes 

network communication means for communicating data server, said method comprising the steps of: 

via a network; 50 displaying a representation of a document and associated 

a processor suitably programmed to run; notes on one of said notes clients; 

a web browser; and generating one or more notes events in accordance with 

a client note plug-in adapted to run within said web user annotations of said document, wherein any num- 

browser. 5^ ber of users in said network may simultaneously anno- 

75. The system according to claim 73, wherein said tate their respective representations of said document; 
second network comprises the internet. synchronizing said one of said notes clients with said 

76. The system according to claim 73, wherein said notes server wherein said one or more notes events are 
second network comprises a TCP/IP based network. sent from said one of said notes clients to said notes 

77. The system according to claim 73, wherein said server, merged with a current notes list maintained by 
second network comprises an Intranet. said notes server and an updated notes list is sent from 

78. The system according to claim 73, wherein said notes said notes server to said one of said notes clients; and 
server comprises: wherein said updated notes list incorporates annotations 

network communication means for communicating data previously made by all other notes clients that have 

via a network; 65 synchronized with said notes server since synchroni- 

a processor suitably programmed to: zation between said one of said notes clients and said 

provide web server services; notes server was last performed. 
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93. The method according to claim 92, wherein said notes 
clients perform synchronization events with said notes 
server asynchronously with respect to each other. 

94. A method of collaborative document annotation for 
use on a notes server in a system including one or more notes 5 
clients, said method comprising the steps of: 

first receiving a request to view a document and in 
response thereto sending requested document data to a 
notes client; 

second receiving a request to retrieve notes associated "^^ 
with said document and in response thereto sending 
requested annotation data to said notes client; 

third receiving one or more notes events associated with 
said document from said notes client in response to a 
synchronization event and in response, merging said 
one or more notes events with a current set of notes 
stored in a notes database and associated with said 
document, generating an updated notes list therefrom 
and sending said updated notes list to said notes client; 
and 

wherein said updated notes list incorporates annotations 
previously made by all other notes clients that have 
synchronized with said notes server since synchroni- 
zation between said one of said notes clients and said 25 
notes server was last performed. 

95. llie method according to claim 94, wherein synchro- 
nization events between said notes clients and said notes 
server occur asynchronously with respect to each other 
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96. A computer program product for use in a notes server, 
said computer program product comprising: 

a computer useable medium having computer readable 
program code means embodied in said medium for 
performing collaboration annotation of documents, 
said computer program product comprising: 

computer readable program code means for receiving a 
request to retrieve notes associated with a document 
and in response thereto sending requested annotation 
data to a notes client; 

computer readable program code means for receiving one 
or more notes events associated with said document 
from said notes client in response to a synchronization 
event; and 

computer readable program code means for merging said 
one or more notes events with a current set of notes 
stored in a notes database and associated with said 
document, generating an updated notes list therefrom 
and sending said updated notes list to said notes client 
in response to said synchronization event, and wherein 
said updated notes list incorporates annotations previ- 
ously made by all other notes clients that have syn- 
chronized with said notes server since synchronization 
between said one of said notes clients and said notes 
server was last performed. 
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